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On  February  29, 1996,  we  testified  on  the  National  Weather  Service’s  (nws) 
Advanced  Weather  Interactive  Processing  System  (awips) — the 
cornerstone  of  nws’  $4.5  billion  modernization  program — before  the 
Subcommittee  on  Energy  and  Environment,  House  Committee  on  Science.1 
This  report  officially  transmits  recommendations  we  made  as  part  of  our 
testimony  which,  if  implemented,  will  strengthen  nws’  ability  to  achieve  a 
fair  return  on  its  awips  investment. 


First,  as  we  also  recommended  in  our  February  1996  report2  on  awips,  nws 
should  (1)  expand  ongoing  awips  requirements  review  activities  to  include 
validation  that  proposed  capabilities  are  justified  on  the  basis  of  mission 
impact  and  (2)  implement  only  those  capabilities  that  are  validated.  These 
recommendations  are  grounded  in  nws’  long-standing  assertion  that  the 
modernization  and  its  component  systems  will  produce  significant  public 
service  improvements — namely,  better  forecasts  at  reduced  costs.  To 
achieve  these  goals,  nws  has  specified  that  awips  must  provide  about  450 
high-order  capabilities,  such  as  the  ability  to  execute  certain  models  or 
display  data  in  certain  formats  and  colors.  These  high-order  capabilities 
are  composed  of  about  22,000  individual  system  requirements. 
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We  testified  that  while  nws  has  done  a  commendable  job  of  incorporating 
the  views  of  the  forecasting  community  in  defining  awips’  capabilities,  it 
has  not  demonstrated  that  all  proposed  capabilities  will  produce  expected 
mission  impacts.  We  have  found  that  successful  public  and  private 
organizations  tie  decisions  on  information  technology  investments  to 
explicit  and  quantifiable  mission  improvements.3  By  failing  to  do  the  same, 
nws  risks  building  either  more  or  less  into  awips  than  necessary,  which, 
either  way,  translates  into  lowering  awips’  retum-on-investment. 


LO 


leather  Forecasting:  New  Processing  System  Faces  Uncertainties  and  Risks  (GAO/T-AIMD-96-47, 
February  29, 1996). 

2Weather  Forecasting:  NWS  Has  Not  Demonstrated  That  New  Processing  System  Will  Improve  Mission 
Effectiveness  (GAO/AIMD-96-29,  February  29, 1996).  “ 

3 Executive  Guide:  Improving  Mission  Performance  Through  Strategic  Information  Management  and 
Technology  (GAO/AIMD-94-115,  May  1994). 


Page  1  GAO/AIMD-96-74  Weather  Forecasting  Recommendations 

uric  quALm  iNcjjt'iiigiiiii)  i 


B-271478 


Second,  we  testified  that  nws  has  wisely  chosen  to  break  awips  software 
development  into  increments,  which  is  a  widely  accepted  means  of 
reducing  software  development  risks.  However,  we  believe  that  nws’ 
approach  to  implementing  this  strategy  makes  awips  development 
unnecessarily  risky  because  nws  chose  to  overlap  development  of  these 
builds,  beginning  development  of  a  new  build  before  the  prior  build  is  fully 
stabilized.  In  doing  so,  it  increased  the  chances  of  potential  defects  being 
passed  along  module-to-module.  Accordingly,  we  recommended  that  nws 
ensure  that  each  build  is  fully  tested  and  all  material  defects  are  corrected 
before  software  development  associated  with  the  next  build  begins. 

Third,  we  recommended  in  our  testimony  that  nws  establish  a  software 
quality  assurance  program  to  increase  the  probability  of  delivering 
promised  awips  capability  on  time  and  within  budget.  While  nws  has 
reported  a  number  of  software  development  process  improvements, 
including  preparing  a  software  development  plan,  we  believe  its  failure  to 
establish  a  software  quality  assurance  program  to  monitor  nws’  in-house 
development  processes  and  products  unnecessarily  puts  awips’  current 
cost,  schedule,  and  performance  goals  in  jeopardy.  Such  a  program  would 
provide  independent  assurance  that  software  development  processes  and 
products  meet  prescribed  standards  and  that  any  identified  deficiencies  in 
process  or  product  are  brought  to  management’s  attention  sooner  rather 
than  later,  thus  minimizing  costly  system  rework.  Employing  a  software 
quality  assurance  program  is  a  common  practice  among  public  and  private 
organizations  with  solid  reputations  for  producing  quality  software. 

Finally,  in  our  testimony,  we  recommended  that  nws  obtain  an 
independent  assessment  of  the  cost  to  develop  and  deploy  awips.  We 
testified  that  nws  currently  does  not  have  reliable  cost  information  on 
awips  development  and  deployment  costs  and  that  its  current  estimate  of 
$525  million,  which  has  not  changed  for  more  than  1  year,  omits  certain 
costs  and  is  thus  understated.  For  example: 

•  The  $525  million  estimate  includes  neither  known  deployment  phase 
contract  cost  increases  nor  nws  internal  program  management  costs 
resulting  from  nws’  1995  decision  to  extend  awips  deployment  1  year.  In 
our  discussions  with  nws  officials,  contract  costs  associated  with  the 
extension  were  estimated  to  be  about  $9  million,  while  nws  program 
management  costs  for  the  additional  year  had  yet  to  be  estimated. 

•  The  estimate  omits  a  known  cost  increase  in  the  development  phase 
contract  of  about  $10  million.  While  nws  officials  stated  that  this  increase 
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will  be  offset  by  reductions  in  the  deployment  phase  contract,  the  terms  of 
the  deployment  phase  have  yet  to  be  renegotiated. 

.  The  estimate  omits  the  cost  of  awips  product  improvements  that  were 
recently  identified  and  are  planned  for  1999.  These  improvements  are 
estimated  to  cost  about  $2  million. 

•  The  $525  million  estimate,  when  originally  derived,  included  about 
$49  million  for  future  development  and  deployment  phase  contract 
modifications — a  $34  million  decrease  from  the  amount  nws  included  in  its 
1992  project  cost  estimate.  Since  1994,  however,  nws  officials  told  us  that 
more  than  $42  million  of  the  $49  million  in  reserve  has  been  allocated  to 
known  contract  costs,  leaving  relatively  meager  reserves  for  inevitable 
modifications.  Omitting  adequate  reserves  at  this  stage  in  awips’ 
development  cycle  is  inconsistent  with  prudent  software  engineering 
practices.  According  to  software  engineering  experts,  the  size  of  a 
system’s  software  at  this  stage  in  its  development  life  cycle  could  grow  by 
as  much  as  50  percent 

As  we  testified,  nws  has  an  opportunity  with  awips  to  substantially  improve 
its  weather  forecasting  capability.  If  the  recommendations  we  made  in  our 
testimony  and  in  this  report  are  implemented,  NWS  can  position  itself  to 
take  full  advantage  of  this  opportunity. 


Agency  Comments 
and  Our  Evaluation 


In  its  written  comments  on  a  draft  of  this  report,  the  National  Oceanic  and 
Atmospheric  Administration  (noaa)  generally  disagreed  with  our 
conclusions  and  recommendations.  A  summary  of  noaa’s  specific 
comments  and  our  evaluation  of  them  follows.  We  have  incorporated 
other  noaa  comments  in  the  report  where  appropriate. 

noaa  disagreed  with  our  first  recommendation  to  (1)  expand  ongoing  awips 
validation  activities  to  include  justifying  proposed  capabilities  on  the  basis 
of  mission  impact  and  (2)  implement  only  mission-justified  capabilities. 
According  to  noaa,  completed  and  ongoing  requirements  reviews  and  risk 
reduction  activities  as  well  as  operational  test  and  evaluation  of  each  awips 
release  are  sufficient  to  ensure  that  unneeded  awips  capabilities  are 
revised  or  not  implemented.  In  addition,  noaa  added  its  view  that 
validating  capabilities  to  mission  impact  is  not  efficient  or  cost-effective 
and  would  cause  further  delay  and  expense  to  the  program.  We  disagree. 
While  we  reiterate  our  commendation  for  awips  requirements  reviews  and 
risk  reduction  activities  to  date  and  support  plans  to  prototype  future 
system  releases,  the  fact  remains  that  none  of  these  activities  were  or  are 
intended  to  demonstrate  the  mission  impact  of  awips  capabilities. 


Page  3 


GAO/AIMD-9 6- 74  Weather  Forecasting  Recommendations 


B-271478 


Moreover,  waiting  until  the  operational  test  and  evaluation  stage  of  each 
awips  release  is  too  late  to  assess  mission  impact  because  by  this  time 
considerable  sums  of  money  will  have  already  been  invested  in  the 
capabilities’  development.  Additionally,  while  validating  capabilities  to 
mission  impact  would  require  some  resource  investment  now,  it  could  also 
prevent  time  and  money  being  spent  unnecessarily  on  developing  awips 
capabilities  that  promise  little  in  terms  of  mission  improvement.  Thus,  we 
continue  to  support  our  recommendation. 

noaa  did  not  comment  specifically  on  our  second  recommendation  to  fully 
test  and  correct  all  material  defects  in  each  awips  build  before  software 
development  associated  with  a  succeeding  build  begins.  However,  noaa 
stated  that  its  plans  for  overlapping  awips’  builds  does  not  introduce 
unnecessary  risks  and  stated  that  judicious  overlapping  is  efficient  and 
appropriate  for  those  builds  that  do  not  need  to  wait  for  the  previous  build 
to  be  “frozen.”  Further,  while  acknowledging  the  potential  risks  of  awips 
build  overlaps,  it  noted  two  factors  that  mitigate  these  risks.  First,  noaa 
stated  that  there  is  minimal  overlap  associated  with  builds  1  and  2  and  that 
the  two  areas  in  which  overlap  occurs  (data  modeling  and 
communications  software)  are  inherently  stable  because  of  early 
development  and  demonstration  activities.  Second,  noaa  stated  that 
development  in  the  later  builds  is  primarily  hydrometeorological 
applications  and  that  these  applications  will  not  interact  directly  with  each 
other.  We  agree  that  overlapping  builds  can  be  appropriate  and  efficient, 
and  that  “freezing”  one  build  before  moving  to  the  next  is  not  always 
efficient.  Further,  we  are  not  questioning  the  degree  of  interaction  among 
hydrometeorological  applications.  Our  concern  is  that  awips  plans  call  for 
extending  or  adding  to  baseline  applications,  such  as  those  that  process 
radar  or  satellite  products  for  display  on  awips  monitors,  in  succeeding 
builds  before  these  baselines  have  even  begun  formal  testing — much  less 
before  the  build  has  been  stabilized — by  removing  all  material  defects.  By 
not  doing  so,  nws  runs  the  risk  of  introducing  defects  on  top  of  unresolved 
defects.  Thus,  we  continue  to  support  our  recommendation  that  nws  fully 
test  each  awips  build  and  correct  all  material  defects  before  beginning 
software  development  associated  with  the  next  build. 

Although  noaa  did  not  directly  address  our  third  recommendation  to 
establish  an  awips  quality  assurance  program,  it  agreed  that  software 
quality  assurance  is  important  for  nws  in-house  software  development  (i.e., 
hydrometeorological  applications),  and  it  stated  that  nws  has  hired  a 
computer  specialist  for  the  awips  program  manager’s  staff  to  oversee  the 
government’s  software  development  activities,  noaa  also  stated  that  nws  is 
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following  accepted  software  development  practices  and  that  an 
independent  review  team  draft  report  gives  a  “generally  favorable 
impression  of  the  current  development  methodology.”  Despite  nws’  recent 
actions,  it  is  important  to  implement  our  recommendation  for  two  reasons. 
First,  hiring  a  computer  specialist  falls  far  short  of  our  recommendation 
because  such  action  does  not  provide  either  the  structure  or  content  of  a 
disciplined,  thorough,  or  independent  quality  assurance  program.  Second, 
since  noaa  provides  no  evidence  or  details  of  actions  taken  to  follow 
“accepted  software  development  practices”  and  no  basis  for  the  “generally 
favorable  impression”  of  its  development  methodology,  these  points  are 
unconvincing.  Thus,  we  continue  to  support  our  recommendation. 

Finally,  noaa  disagreed  with  our  fourth  recommendation  to  obtain  an 
independent  assessment  of  the  cost  to  develop  and  deploy  awips,  adding 
that  it  plans  to  reevaluate  its  current  cost  estimate  of  $525  million  in 
August  1996  when  the  deployment  phase  contract  is  renegotiated  and  that 
it  anticipates  no  significant  increases  in  the  current  estimate,  noaa  further 
stated  that  developing  a  new  estimate  at  this  time  would  add  little  value  to 
program  and  budget  planning.  In  defending  the  validity  of  its  current 
estimate  of  $525  million,  noaa  also  dismissed  each  of  the  omitted  cost 
items  cited  in  our  report  for  reasons  cited  below.  We  disagree  that  a  new 
estimate  would  add  little  value,  particularly  in  light  of  expressed  interest 
by  members  and  staff  of  the  Subcommittee  on  Energy  and  Environment, 
House  Committee  on  Science,  in  having  an  up-to-date,  reliable  estimate 
upon  which  to  base  authorization  decisions.  Because  noaa  did  not 
acknowledge  that  these  costs  should  be  included  in  their  project  cost 
estimate,  we  believe  our  recommendation  remains  valid,  noaa’s  comments 
on  each  of  the  omitted  cost  items  cited  in  our  report  and  our  response 
follow. 

•  noaa  stated  that  the  $9  million  increase  associated  with  extending  the 
deployment  phase  by  1  year  should  not  be  added  to  the  awips  project  cost 
estimate  because  nws  is  committed  to  completing  the  deployment  phase  in 
24  months  as  originally  planned.  This  contradicts  nws  documentation 
dated  February  9, 1996,  stating  that  the  awips  deployment  phase  would  be 
extended  1  year  and  would  result  in  an  additional  $9  million,  and  noaa  has 
provided  no  documentation  rescinding  this  plan  or  reassessing  its  cost. 

•  noaa  repeats  its  claim  that  the  omission  of  a  $10  million  cost  increase  to 
the  development  phase  contract  will  be  offset  by  a  corresponding 
decrease  in  the  deployment  phase  contract.  However,  as  stated  in  our 
report,  since  the  deployment  phase  contract  has  yet  to  be  renegotiated, 
there  is  no  basis  for  noaa’s  claim. 
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•  noaa  stated  that  the  $2  million  omission  for  recently  identified  product 
improvements  should  not  be  included  in  the  awips  cost  estimate  because 
this  awips  enhancement  will  be  developed  after  the  deployment  phase  is 
completed.  We  disagree.  Known  system  development  costs  cannot 
rationally  be  excluded  from  cost  estimates.  Moreover,  the  awips  project 
cost  estimate  already  includes  numerous  other  awips  product 
improvements  and  thus  excluding  the  cost  of  this  product  improvement 
would  be  inconsistent  with  nws’  own  practice. 

•  noaa  acknowledged  that  the  $525  million  estimate  provides  a  relatively 
small  reserve  for  contract  modifications  needed  to  address  such  things  as 
code  growth.  However,  noaa  contends  that  inherent  conservatism  in  the 
deployment  phase  component  of  the  estimate  will  ultimately  produce 
more  contingency  money  than  is  currently  apparent  in  the  estimate.  Also, 
noaa  stated  that  it  does  not  expect  major  modifications  for  the  remainder 
of  the  development  phase.  Despite  noaa’s  optimism,  it  admits  that  contract 
costs  (development  and  deployment)  are  the  greatest  variable  in  the 
current  cost  estimate.  Further,  prudent  software  engineering  practices 
recommend  planning  for  as  much  as  50  percent  growth  in  software  size  for 
systems  at  awips’  stage  of  development.  Finally,  federal  agencies’  past 
performance  in  acquiring  major,  software  intensive  systems  is  routinely 
characterized  by  high  cost  overruns  and  significant  schedule  slippage. 
Thus,  we  find  no  basis  for  noaa  to  conclude  that  contingency  money  will 
appear  before  the  deployment  phase  contract  is  renegotiated. 

The  full  text  of  noaa’s  comments  is  provided  in  appendix  I. 


This  report  contains  recommendations  to  you.  The  head  of  a  federal 
agency  is  required  by  31  U.S.C.  720  to  submit  a  written  statement  on 
actions  taken  on  these  recommendations  to  the  Senate  Committee  on 
Governmental  Affairs  and  the  House  Committee  on  Government  Reform 
and  Oversight  not  later  than  60  days  after  the  date  of  this  report.  A  written 
statement  also  must  be  sent  to  the  House  and  Senate  Committees  on 
Appropriations  with  the  agency’s  first  request  for  appropriations  made 
more  than  60  days  after  the  date  of  this  report. 
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We  are  sending  copies  of  this  letter  to  each  member  of  the  Subcommittee 
on  Energy  and  Environment,  House  Committee  on  Science,  and  other 
interested  parties.  We  will  send  copies  to  others  upon  request.  If  you  have 
questions  or  wish  to  discuss  the  issues  in  this  report  further,  please 
contact  me  at  (202)  512-6240.  Major  contributors  to  this  report  are  listed  in 
appendix  H 

Sincerely  yours, 


Jack  L.  Brock,  Jr. 

Director,  Information  Resources  Management/ 

Resources,  Community,  and  Economic  Development 
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Comments  From  the  National  Oceanic  and 
Atmospheric  Administration 


Note:  GAO  comments 
supplementing  those  in  the 
report  text  appear  at  the 
end  of  this  appendix. 
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UNITED  STATES  DEPARTMENT  OF  COMMERCE 
The  Under  Secretary  for 
Ocean*  and  Atmosphere 

Washington,  D.C.  20230 

April  17,  1996 


Mr.  Gene  L.  Dodaro 
Assistant  Comptroller  General 
Accounting  and  Information  Management 
Division 

United  States  General  Accounting  Office 
Washington,  D.C.  20548 

Dear  Mr.  Dodaro: 

Enclosed  is  the  Department  of  Commerce's  reply  to  the 
General  Accounting  Office  draft  report:  Weather  Forecasting: 
Recommendations  to  Address  New  Weather  Processing  System 
Development  Risks  (GAO/AIMD-96-74) . 

These  comments  were  prepared  in  accordance  with  the  Office 
of  Management  Budget  Circular  A-50. 


Sincerely, 


D.  James  Baker 


Enclosure 


THE  ADMINISTRATOR 
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See  comment  1. 


See  comment  1. 


The  National  Oceanic  and  Atmospheric  Administration's  (NOAA) 
Response  to  the  General  Accounting  Office  (GAO)  Draft  Report: 
"Weather  Forecasting:  Recommendations  to  Address  New  Weather 
Processing  System  Development  Risks  (GAO/AIMD-96-74) f "  dated 
April  1996. 

Recommendation  1:  * First ,  as  we  recommended  in  our  February  1996 
report  on  the  Advanced  Weather  Information  Processing  System 
(AWIPS) ,  the  National  Weather  Service  (NWS)  should  (1)  expand 
ongoing  AWIPS  requirements  review  activities  to  include 
validation  that  proposed  capabilities  are  justified  on  the  basis 
of  mission  impact  and  (2)  implement  only  those  capabilities  that 
are  validated . " 

NOAA  prepared  a  formal  response  to  these  recommendations 
made  in  the  GAO  February  1996  report  on  AWIPS.  We  do  not 
agree  with  them.  The  best,  and  often  only,  method  for 
validating  the  proposed  capabilities  is  to  continue  with 
planned  risk  reduction  activities  and  the  operational  test 
and  evaluation  of  each  system  release.  Both  these 
activities  will  provide  vital  information  back  into  the 
development  process  so  we  can  be  assured  that  the  system 
capabilities  are  validated.  This  process  will  ensure  that 
capabilities  that  are  not  needed  will  be  revised  or  not 
implemented. 

Expending  the  additional  time  and  resources  required  to 
validate  all  capabilities  on  the  basis  of  mission  impact 
in  advance  of  incremental  deployment  is  not  an  efficient 
or  cost-effective  process.  If  implemented,  the  recommended 
actions  would  cause  further  delay  and  expense  to  the 
program  and  yield  no  significant  reduction  in  program 
risk.  The  current  risk  that  resources  will  be  devoted  to 
developing,  testing  and  deploying  unnecessary  capabilities 
is  negligible.  The  risk  that  the  NWS  will  not  be  able  to 
maintain  operations  with  archaic  systems  without  AWIPS  is 
high.  The  inability  of  NWS  to  capitalize  on  powerful  new 
information  services  already  available  from  the 
modernization  without  AWIPS  is  certain.  We  are  confident 
that  the  requirements  reviews  and  risk  reduction  activities 
mentioned  in  the  GAO  report  and  ongoing  risk  reduction  and 
operational  test  and  evaluation  activities  are  sufficient  to 
validate  AWIPS  capabilities. 

Recommendation  2:  m  Accordingly,  we  recommended  that  the  NWS 
ensure  that  each  build  is  fully  tested  and  all  material  defects 
are  corrected  before  software  development  associated  with  the 
next  build  begins . " 

NOAA  does  not  agree  that  the  overlapping  build  approach 
employed  for  AWIPS  development  introduces  unnecessary  risk. 
As  identified  in  the  GAO  report,  incremental  development  is 
a  common  approach  used  throughout  the  software  industry  for 
reducing  development  risks.  Throughout  the  AWIPS 
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development  life-cycle  we  propose  to  follow  this 
risk  -reducing  approach.  GAO  does  not  take  exception  to 
the  proposed  incremental  nature  of  AWIPS  development,  but 
only  to  the  proposed  overlap  of  AWIPS'  software  builds. 

There  are  two  aspects  of  the  incremental  approach  being 
incorporated  into  the  AWIPS  development  which  mitigate  these 
potential  risks. 

First,  much  of  the  basic  system  infrastructure  is  being 
developed  in  the  early  builds  (1  and  2),  during  which 
overlap  is  not  being  generally  applied.  The  only  overlaps 
of  activity  between  Builds  1  and  2  are  in  the  areas  of  data 
modeling  and  enhancements  to  some  communications  software. 
Build  2  data  modeling  provides  information  to  fill  in 
build-specific  details  based  on  a  stable  overall  AWIPS 
conceptual  data  model,  developed  during  the  summer  and  fall 
of  1995  during  an  extended  set  of  sessions  between  PRC  data 
modelers  and  government  data  experts.  This  build-specific 
data  modeling  activity  is  an  analysis  step  which  must  be 
completed  before  any  software  upgrades  to  data  management 
modules  can  be  made  to  meet  requirements  for  data  access  by 
Build  2  applications  through  Build  2  data  objects. 

The  communications  software  components  for  Build  1  are, 
to  a  significant  degree,  taken  from  the  long-lead-time 
satellite  broadcast  network  software  components  implemented 
and  tested  operationally  during  the  Pathfinder  development 
and  deployment.  Thus,  stable  functionality  of  the  Build  1 
communications  software  has  already  been  demonstrated 
operationally.  This  stability  has  allowed  the 
communications  developers  to  move  into  Build  2  detailed 
design  and  development  activities  prior  to  the  formal 
acceptance  of  Build  1. 

Second,  much  of  the  development  in  later  builds  will  be 
in  hydrometeorological  (HM)  modules  which  use  the  system 
infrastructure.  Many  HM  applications  will  be  "display 
oriented"  and  will  not  interact  directly  with  each  other. 

The  main  coupling  among  the  HM  applications  is  through 
the  organization  of  data  in  the  AWIPS  database.  As  with 
Build  2,  an  early  data  modeling  activity  will  benefit  every 
AWIPS  build  so  that  the  data  needed  to  test  applications 
would  be  available  in  a  timely  fashion  and  would  allow  for 
smooth  integration  of  HM  applications  within  any  given 
build's  development  cycle. 

Because  of  the  way  AWIPS  functionality  is  evolving 
throughout  the  program  life-cycle,  an  incremental,  low-risk 
approach  that  allows  judicious  overlapping  is  efficient  and 
appropriate  for  those  build  components  that  do  not  need  to 
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wait  for  the  previous  build  configuration  to  be  frozen.  In 
the  case  of  data  modeling,  it  is  a  necessary  first  step  to 
any  build's  development  and  must  be  started  early  to  avoid 
costly  delays  in  delivery  of  the  incremental  AWIPS 
capabilities  that  will  be  growing  with  each  build. 

Recommendation  3:  "Third,  we  recommended  in  our  testimony  that 
ms  establish  a  software  quality  assurance  program  to  increase 
the  probability  of  delivering  promised  AWIPS  capability  on  time 
and  within  budget 

NOAA  agrees  that  software  quality  assurance  is  important  for 
See  comment!  the  hydrometeorological  applications.  A  computer  specialist 

has  been  hired  for  the  AWIPS  Program  Manager's  staff  to 
oversee  software  development  activities,  including 
government  developed  software,  from  a  program  standpoint. 
Quality  assurance  is  one  of  the  responsibilities  of  this 
position. 

Accepted  practices  of  software  development  are  being  used  in 
the  development  of  hydrometeorological  applications.  Peer 
reviews  and  code  walkthroughs  are  standard.  Formal  testing 
is  conducted  by  an  independent  designated  tester  and 
regression  tests  are  performed  as  necessary.  Testing  will 
be  performed  on  the  NWS  Headquarters  Test  and  Integration 
Facility,  an  AWIPS  system,  which  is  kept  under  project 
configuration  control.  Trouble  reports  are  generated  and 
tracked. 

An  independent  review  team  recently  studied  the  Government's 
software  development  process  at  the  request  of  the 
Department  of  Commerce.  In  their  draft  report,  the  team 
gave  a  "generally  favorable  impression  of  the  current 
development  methodology."  Their  primary  concerns  in  the 
area  of  software  development  related  to  development  at  a 
local  level  and  how  the  methodology  for  doing  this  would  be 
established. 

Recommendation  4:  * Finally ,  in  our  testimony ,  we  recommended  that 
NWS  obtain  an  independent  assessment  of  the  cost  to  develop  and 
deploy  AWIPS," 

See  comment  1  NOAA  does  not  that  an  independent  assessment  of  costs 

for  the  development  and  deployment  of  AWIPS  should  be 
conducted.  The  current  estimate  at  completion  (EAC)  for  the 
development  and  deployment  of  AWIPS  is  $525  million.  This 
estimate  was  developed  in  December  1994  on  the  basis  of 
Government  estimates  of  contract  costs  for  both  the 
Development  and  Deployment  Phases  in  the  restructured 
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program.  Since  then,  the  Development  Phase  of  the  contract 
has  been  renegotiated  and  NOAA  will  enter  into  negotiations 
later  this  month  on  the  contract  modification  for  the 
Deployment  Phase.  We  recently  completed  an  independent 
Government  cost  estimate  for  the  Deployment  Phase,  and  in 
that  context,  reviewed  the  total  Development  and  Deployment 
Phase  costs  contained  in  the  EAC.  We  do  not  anticipate  a 
significant  increase  overall  for  these  two  phases.  By 
August  1996,  we  will  have  negotiated  costs  for  both  phases 
and  will  reevaluate  the  EAC  at  that  time.  Since  contract 
costs  are  the  greatest  variable  in  the  EAC,  developing  a  new 
estimate  at  this  time  would  add  little  value  to  program  and 
budget  planning. 

The  GAO  report  states  that  there  was  a  decision  to  extend 
AWIPS  deployment  for  one  year.  NOAA  is  proceeding  with  a 
24  month  deployment  schedule,  as  planned.  The  additional 
1-year  window  is  included  in  the  request  for  proposal  to  PRC 
regarding  the  modification  to  the  Deployment  Phase.  This 
contract  structure  is  being  considered  to  provide 
contractual  flexibility  in  the  event  of  funding  shortfalls 
in  FY  1997  and  FY  1998.  PRC  has  been  asked  to  price  this 
modification  based  upon  a  deployment  schedule  that  is 
completed  30-months  after  the  start  of  the  Deployment  Phase. 
The  $9  million  increase  quoted  by  GAO  is  based  on  this 
30-month  schedule.  NOAA  is  committed  to  completing  the 
deployment  in  24  months  and  has  focused  all  planning, 
scheduling,  and  resources  of  the  AWIPS  program  to  meet  this 
goal.  The  EAC  will  not  be  revised  to  accommodate  the 
30-month  schedule  unless,  and  until,  NOAA  revises  its  plan. 

The  GAO  report  references  $10  million  in  Development  Phase 
costs  which  were  not  accounted  for  in  the  1994  EAC.  These 
are  costs  which  were  originally  included  in  the  Deployment 
Phase  estimate.  They  will  be  paid  either  in  the  Development 
Phase  or  the  Deployment  Phase,  but  not  in  both.  Therefore, 
this  increase  in  the  Development  Phase  costs  will  result  in 
a  corresponding  decrease  in  the  deployment  phase  costs. 

The  GAO  report  states  that  $2  million  for  product 
improvement  is  not  included  in  the  EAC.  The  AWIPS  program 
is  planning  for  future  product  improvements  after  the 
completion  of  the  Deployment  Phase.  The  resources  planned 
for  this  effort  should  not  be  included  in  the  EAC  for  the 
program,  as  the  EAC  represents  costs  only  through  the  end  of 
nationwide  deployment.  The  resources  for  this  effort 
represent  life  cycle  costs  of  AWIPS,  similar  to  operations 
and  maintenance  costs. 
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The  GAO  report  states  that  the  EAC  does  not  include 
additional  costs  associated  with  the  Government’s 
responsibility  for  developing  all  hydrometeorological 
applications.  This  is  not  correct.  When  the  $525  million 
See  comment  2  estimate  was  established  in  1994,  the  design  of  the 

restructured  program  was  known,  including  Government 
responsibility  for  hydrometeorological  applications.  This 
responsibility  was  accounted  for  in  the  1994  EAC  and  is 
included  in  the  $108  million  estimate  for  Government 
Activities. 

The  GAO  report  raises  concerns  that  the  program  does  not 
have  adequate  reserves.  Although  the  amount  of  reserve 
available  for  contract  modifications  is  relatively  small,  we 
believe  that  our  deployment  phase  estimates  are  conservative 
and  that  there  will  be  more  contingency  in  the  EAC  than  is 
now  apparent.  Furthermore,  since  the  restructured 
Development  Phase  contract  incorporates  most  of  the 
additional  requirements  envisioned  by  the  program,  we  do  not 
anticipate  major  modifications  for  the  remainder  of  the 
Development  Phase. 
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The  following  are  gao’s  comments  on  the  National  Oceanic  and 
Atmospheric  Administration’s  letter  dated  April  17, 1996. 


GAO  Comments 


1.  Discussed  in  the  “Agency  Comments  and  Our  Evaluation”  section  of  the 
report. 

2.  Discussion  of  this  point  deleted. 
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